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This series of reports is designed to support the planning and implementation of the Maternal 
and Child Health Bureau (MCHB) State Early Childhood Comprehensive Systems (SECCS) 
Initiative. The series was edited by Neal Halfon, Thomas Riee, and Moira Inkelas. The reports 
were written by a team of experts to provide guidanee on state poliey development within the 
SECCS Initiative. Poliey reports on erosseutting themes include strategic planning, 
communications strategies, financing, results-based accountability, cultural proficiency, and data 
analysis and use. Policy reports on programmatic topics include medical home, parenting 
education, family support, infant mental health, and dental health. 

This work was conducted as part of a cooperative agreement to the National Center for Infant 
and Early Childhood Health Policy from the Health Resources and Services Administration 
(HRSA), MCHB, 5U05-MC00001-02. 

The National Center for Infant and Early Childhood Health Policy supports the federal 
MCHB and the SECCS Initiative by synthesizing the policy relevance of important and 
emerging early childhood health issues, conducting policy analysis on systems-building and 
programmatic issues, and disseminating the latest research findings to increase the visibility of 
early childhood policy issues on the national agenda. The National Center for Infant and Early 
Childhood Health Policy is a partnership of the UCEA Center for Healthier Children, Eamilies 
and Communities; The Women's and Children's Health Policy Center of the Johns Hopkins 
Bloomberg School of Public Health; and the Association of Maternal and Child Health 
Programs. 
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Executive Summary 



The federal Maternal and Child Health Bureau has launched a five-year initiative that will 
support state efforts to build comprehensive early childhood service systems. This initiative - the 
State Early Childhood Comprehensive Systems Initiative (SECCS) - provides two year planning 
grants followed by three year implementation grants to the 50 state Maternal and Child Health 
agencies. The purpose of these grants is to help coordinate, integrate and improve the access to, 
and the quality of, health, early education, parent education, and family support services for 
young children and their families. The Initiative seeks to help assure that all children enter 
school healthy and ready to learn, and achieve the fullest possible social and emotional 
development. 

This paper sets forth a disciplined thinking process designed to do these things — to help identify 
actions that will measurably improve the lives of children, families and communities. It is now 
being used, in whole or in part, in at least 40 states and eight countries. It breaks with past 
planning methods in several important ways. Eirst it begins with discipline about language and 
the use of words to label ideas. Second, it posits a sharp distinction between accountability for 
the well-being of whole populations and accountability for the performance of programs, 
agencies and service systems. And third it offers a common sense progression of work, from talk 
to action, that produces effective actions with minimum paper. While the following sections 
focus on the well-being of young children and their families, this framework is being applied to 
the well-being of many other groups, up to and including the well-being of whole populations in 
cities, counties, states and nations. The entire framework can be found on the website: 
http://www.raguide.org. 

SECCS use of this framework begins with the population of all young children and their families 
in the state. What results do we want for this population (e.g., all children healthy and ready for 
school) and what are the most powerful indicators that tell us if we are achieving these results, 
like the rate of low-birthweight births, rate of immunization, and reading scores in the early 
grades? Baselines for these indicators, with history and forecast, are next used to tell if 
conditions are getting better or worse. They show where we’ve been and where we’re headed if 
we stay on our current course. Success can be gauged by how well we do against these baselines. 
By examining the story behind the baselines, we can understand why the numbers look the way 
they do, the causes and forces at work in shaping the well-being of young children and their 
families. Next, we identify the partners who have a potential role to play in doing better, and 
what works - what it would take - to measurably improve these conditions. Thinking about 
“what works” should draw on research and experience in other communities, but must not be 
limited by research. We need to use our personal experience and knowledge of our communities 
to decide what would work here to make a difference. This thinking should also include no-cost 
and low cost actions, and the contributions of many partners. An action plan and budget can then 
be constructed from the most powerful of these ideas, those that are consonant with community 
values and that have the greatest chance to make a difference. Partners’ resources can then be 
used, in combination with other resources in the community, to move forward with 
implementation of the plan. 
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Using this framework, the SECCS Initiative ean beeome not just another planning proeess, but 
rather a chance to galvanize community support for a broad range of actions by many partners to 
improve the well-being of young children and their families. We have an historic opportunity to 
make investments in child and family well-being that will pay off for decades to come. If the 
processes we create are all-talk-and-no-action token efforts, we will not be remembered for using 
this opportunity well. If instead we bring business-like discipline to the demands of improving 
conditions of well-being for young children and their families, then we have a chance to be 
remembered differently. The approach offered in this planning guide can be used to structure the 
planning process, and future iterations of the planning process, to produce the legacy of results 
we want for young children and their families. 
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Introduction 



Each participant in the State Early Childhood Comprehensive Systems Initiative (SECCS)' has 
two types of responsibility. Eirst the state grantee must help craft a strategy to ensure that ALE 
ehildren prenatal to age five enter school in good health, ready and able to learn and soeially and 

emotionally well developed. Second, the Initiative must work to 
assure that the programs, ageneies and serviee systems which 
provide vital serviees to children and families are eoordinated, 
integrated and well managed. 

In short, the states must (help) develop a strategy whieh goes beyond 
the formal delivery of serviees, and also help improve the 
performanee of the serviee delivery system itself. This poliey brief 
will address how states and their partners ean organize their work, 
and just as importantly their thinking, to do both of these things 
well. 



Eventually, all states will be ealled upon to answer the kinds of 
questions addressed in results-based deeision making: “Are ehildren and families better off? 
What differenee did our work make?” The sections that follow can help you do this. If you’re 
still skeptieal, as you should be, then skip to the imaginary press conference at the end of the 
paper. 

Before proeeeding with the planning proeess itself, let’s pause for a word about the matter of 
ambition. It would be easy to eonelude that the limited amount of funding provided in these 
grants is eause for limiting our ambitions about what ean be aeeomplished. This paper starts with 
no sueh limitation. It sets out a method of aehieving what some would eall unaehievable: the 
measurable improvement in the well-being of all ehildren zero to five and the transformation of 
the serviee systems whieh support them. You may rightly make choiees which narrow the scope 
of what you set out to do, and it would be hard to fault you for doing so. But this paper provides 
a framework within which to make those ehoices, and perhaps, when the time is right, to revisit 
what is possible on a larger scale. Please read it with these thoughts in mind. 



Eventually, all states 
will be called upon to 
answer the kinds of 
questions addressed 
in results-based 
decision making: 
“Are children and 
families better off? 
What difference did 
our work make? ” 



The Language of Accountability 

One of the most diffieult problems in this work is the problem of language. People eome to the 
table from many different diseiplines and many different walks of life. And the way in which we 



'The problem with acronyms is that they are a form of exclusionary jargon which makes the work less accessible to 
lay citizens. However if we spelled out State Early Childhood Comprehensive Systems Initiative every time the 
paper would be twice as long. 
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talk about programs and services for children and families varies all over the map. This means 
that the usual state of affairs in planning for children and families is a Tower of Babel, where no 
one really knows what the other person is saying, but everyone politely pretends that they do. As 
a consequence, the work is slow, frustrating and often ineffective. 

It is possible to exercise language discipline in this work. And the 
way to do this is to agree on a set of definitions that start with ideas 
and not words. Words are just labels for ideas. And the same idea 
can have many different labels. The following four ideas are the 
basis for definitions used at the beginning of this work. Alternative 
labels are offered; 

Results (or outcomes or community-wide results or community- 
wide outcomes) are conditions of well-being for children, adults, 
families or communities, stated in plain English (or plain Spanish or 
plain Korean...). Results are plain language statements of quality of 
life conditions that voters and taxpayers can understand. They are 
not about programs or agencies or government jargon. Results 
include such conditions as: “healthy children, children ready for 
school, children succeeding in school, children staying out of 
trouble, strong families, and safe communities." 

Indicators (or benchmarks)^ are measures that help quantify the 
achievement of a result. They answer the question, "How would we recognize these results in 
measurable terms if we fell over them?" So, for example, the rate of low-birthweight babies 
helps quantify whether we're getting healthy births or not. Third grade reading scores help 
quantify whether children are succeeding in school today, and whether they were ready for 
school three years ago."^ The crime rate helps quantify whether we are living in safe 
communities; the air quality index whether we are living in a clean environment and so forth. 

Strategies are coherent collections of actions which have a reasoned chance^ of improving 
results. Strategies are made up of our best thinking about what works, and include the 
contributions of many partners. No single action by any one agency can create the improved 
results we want and need. 



The principle 
distinction is 
between ends and 
means. Results and 
indicators are 
about the ends we 
want for children 
and families. 
Strategies and 
performance 
measures are about 
the means to get 
there. 



2 

See Appendix B “A Tool for Choosing a Common Language” for a method of creating agreement on word usage 
for a much larger set of ideas. 

3 

The word benchmark is used to label this idea in a number of states and counties. But it has a very different 
meaning in the business community where it means the level of achievement of a successful competitor. For this 
reason, “indicator” may the better choice. 

^ 3"* grade reading scores are a lagging indicator of children ready for school, and can be a good proxy for school 
readiness when a kindergarten entry assessment process is not in place. 

^ This reasoning process is sometimes referred to as “logic model,” or “theory of change.” This means, “What is the 
theory that this might actually work?” There is a whole body of writing on this subject. See the reference to the work 
of Kubisch and Connell in Appendix A. 
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Performance Measures (or performance indicators) are measures of how well public and 
private programs, agencies and service systems are working. Performance measures can tell us 
if a program is delivering services well, or if a service system is functioning in an integrated and 
coordinated way. The most important performance measures tell us whether the clients or 
customers of the program, agency or service system are better off. We sometimes refer to these 
measures as client or customer'^ results (to distinguish them from community-wide results for all 
children and families). 



The principal distinction here is between ends and means. Results 
and indicators are about the ends we want for children and families. 
And strategies and performance measures are about the means to get 
there. ^ Processes that fail to make these crucial distinctions often mix 
up ends and means, and tend to get mired in the all-talk-no-action 
circles that have disillusioned countless participants in past efforts. 

States actually have some choices about which labels to attach to 
these ideas. Some of the initiative documents use a wide range of 
terms without always clearly specifying which ideas are being 
referenced. Appendix B provides a Tool for Choosing a Common 
Language, which presents choices about language usage and a place 

o 

to record your decisions. We will use results, indicators, strategies 
and performance measures to label the above ideas in this paper. You 
can use Appendix B to pick whatever labels work for you. 



People bring 
different language 
usage to the table 
from many different 
diseiplines and 
walks of life. It is 
possible to exercise 
language discipline 
in this work by 
agreeing on a set of 
definitions that 
start with ideas 
and not words. 



Responsibility 1: How to help assure that all children are healthy and ready for school 

In this section we will explore a very simple, but business-like, thinking process that can help 
direct the work of creating a strategy for making all children healthy and ready for school. It is 
the thinking process that underlies all of results accountability, whether the population in 
question is children, elders or whales. It is the simple set of notions: “What do we want for 
children prenatal to age five in plain English?” “How would we recognize it in measurable 
terms?” “What will it take to get there?” This process is displayed on the chart in Appendix C. 
In the following paragraphs we will take a quick pass at the basic ideas in each step of the 

^ Or consumer, patient or student results, depending on the service system. 

^ This ends/means distinction actually operates at many different levels at the same time. Results, indicators, 
strategies and performance measures are the highest level. Within programs and agencies, customer results become 
the ends and the actions of the agency or program become the means. Like a set of Russian dolls, ends and means 
are nested together from the highest to the lowest level. 

g 

See http://www.raguide.org/tool_for_choosing_a_common_language.htm for more information on the 
development and use of this tool. 
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thinking process. And then in the next section of the paper we will go back and explore how to 
do eaeh step in more detail. 

What do we want? 

Population : Results aceountability starts with a whole population. In the ease of the SECCS 
Initiative, this is all children, prenatal to age five who live in the state. It also includes, by 
extension, all the families in which these children live, or are about to be born. This is important 
beeause many of the things whieh will work to improve the health and sehool readiness of 
children involve helping their families, and in partieular, their parents, be sueeessful. So SECCS 
is about the well-being of the population of all young ehildren and their families. 

Results : Results are eonditions of well-being we would like to say exist for this population. 
These conditions are stated in plain English (plain Spanish, plain Vietnamese, plain Earsi...) not 
bureaueratic jargon. The Initiative referenees the results: “children enter school healthy and 
ready to learn,” and “young children experience full “soeioemotional development.” These are 
statements that the public can understand,^ and that can be used to eommunicate the basic 
purpose of SECCS and anehor this work. There are of eourse many other results for children and 
families whieh are important (e.g., “All ehildren are safe, all children suceeed in school, families 
are strong and self suffieient.”) SECCS, if successful, will make a eontribution to these other 
results as well. 



How Would We Recognize It? 

The next two questions have to do with how we would 
reeognize these conditions if we fell over them, first in 
terms of experience and then in terms of data. 

Experience : How would we recognize these results in 
our day to day lives in the community? What would 
we see, hear, feel, observe? (e.g.. We would see 
children playing outside. We would hear young 
children with good communication skills. We would 
feel that ehildren were respeeted and loved in our 
community.) 

Indicators : How would we recognize these conditions 
in measurable terms. Here we are looking for data that 
tell us whether these conditions exist or not. If the 
condition is child health, we might look at the rate of 
low birth-weight babies, or the rate of emergency 
room visits. If the result is “children sueceeding in 



Results-Based Decision Making 



Population: Children prenatal to age 5 
Results: Children enter school healthy and 
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Although the phrase “social-emotional development” is dangerously close to jargon. 
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school” we might look at the pereent of ehildren reading at grade level or entering sehool with 
soeial and emotional development appropriate to their age. We start with the data we eurrently 
have and ean use early in the planning proeess. Other data, that we would like to have, or that 
needs to be improved, beeomes part of our data development agenda. 

Baselines : For eaeh indicator, we present a picture of where we've been and where we're headed 
if we stay on our eurrent eourse. These pietures are ealled baselines. They allow us to define 
suecess as doing better than the baseline. 

What will it take to get there? 

Story behind the baselines: Why do these baseline pietures look the way they do? What are the 
eauses and forees at work? This is the epidemiology part of the work. Digging behind the 
pietures helps us get a handle on what’s going on in our eommunity and prepares us for the 
discussion of what might work to do better. As we do this work we bump up against plaees 
where we wish we had more information. This beeomes part of our information/research agenda. 
We’ll gather this information as best we ean between meetings. 

Partners: Who are the potential partners (people and agencies, publie and private) who have a 
role to play in doing better? 

What works : What do we think would work? What would it take to do better than the baselines in 
this eommunity? What has worked in other plaees outside our eommunity? What does the 
researeh tell us? Just importantly, what does our own personal experienee tell us about what 
would work here? The answers should draw on the possible eontributions of a wide range of 
partners, and should involve no-eost and low-eost ideas. The Initiative sets forth a speeifie set of 
ideas about what works: aeeess to medieal homes, family support serviees, parenting edueation, 
soeial and emotional health and development, and quality early eare and edueation - that ean be 
used as a starting point for this step in the thinking proeess. The Initiative also posits that serviee 
systems whieh are eoordinated and integrated are an essential part of what works. 

This systems level thinking is addressed in two places in this paper. Crafting what works ideas 
into a eoherent strategy involves thinking about the design of systems of serviee, and how they 
ean be more than aggregations of programs and ageneies.'^ How ean we make these systems 
more eoordinated and integrated? 

How to manage the performanee of these serviee systems is addressed in the seeond part of this 
paper. It starts with identifying the key systems-level performanee measures and then proeeeds 
through a diagnostie and deeision making proeess similar to population level work whieh leads 
to an aetion plan to improve performanee. Work to improve serviee system performanee (and its 



See Section 5 below: “Craft what works ideas into a coherent strategy” 

See Section 7 below: “Using performance accountability to achieve high quality performance of programs, 
agencies and service systems” 
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attendant customer population) is, in essence, nested inside work to improve the well-being of 
the whole population of all children and families. 

Criteria : If we come up with a long list of things that might work, how do we choose what to 
actually do? What criteria should guide this priority setting process? Some criteria to think about 
include: specificity (Is the idea about specific action not rhetoric?), leverage (Will it make a big 
or little difference?), values (Is it consistent with our personal and community values?), and 
reach (Is it feasible to do it this year, next year or three to 10 years?). 

Action nlan and budget: What do we nronose to actuallv do? This should take the form of a 
multi-year action plan laying out what is to be accomplished by when. We can then assign 
responsibilities and get started. Once you decide on things to be done, projects, programs, no- 
cost and low-cost actions etc., you can use performance measures to assess progress and improve 
performance. 

This does not have to take forever. It is possible to take a pass at this thinking process in an hour 

13 

or so. And then go through it again each time your team and/or your partners get together. 
Every time you iterate this process, the action plan gets better. 

How to do this work 

The sections which follow explore how to do each step in the thinking process outlined above. 
Of course there is no one right way to do planning for SECCS or anything else. This paper 
presents an approach, not the approach to this work. We encourage SECCS states to be good 
consumers of advice, to consider a range of approaches, and select the one that makes the most 
sense to the state and its partners. We suggest you consider the following eight steps in this 
process: 

1 . Get people together. 

2. Choose results and indicators for young children and their families. 

3. Develop baselines and the story behind the baselines. 

4. Consider what works. 

5. Craft what works ideas into a coherent strategy. 

6. Work with your partners to implement that strategy. 

7. Use performance accountability to assure quality of service. 

8. Play it again. i4 

Although this thinking process is presented in linear form for sake of clarity, these steps do not 
have to be followed strictly in sequence. This process is iterative, and much of the work will 
proceed along parallel tracks. Steps which are skipped can be added later or addressed on the 



The words “goals and objectives” are commonly used to label the component parts of action plans. If you do this, 
however, be careful that you do not also attach other meanings to the word goal, such as community conditions of 

well-being. See http://www.raguide.org/the_language_of_accountability.htm 

13 

See the “Turn the Curve Exercises on http://www.raguide.org under “Tools.” 

14 Sam 
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next pass of the planning eycle. 



1. Get people together 



The Initiative Guidanee makes referenee 
to many different partners, and a longer 
list of potential partners is shown in the 
box at the right. 



Potential Early Childhood 
Comprehensive System Partners 

Advocacy organizations Governor 

Behavioral health Health Maintenance 



How could you organize the work so that 
all these disparate voices are heard. 

While the full detail about how to do this 
is beyond the scope of this guide the 
general approach has three parts: 

Create the state systems building 
initiative as the nucleus of a larger set of 
partners. The leadership of the state 
initiative should conceive of itself as a 
core set of partners, not the chosen few. 

And, given the small amount of SECCS 
funding, the core group should think in 
terms of making the case for change, and 
influencing and leveraging actions and 
assets at the community and system 
levels. The core group should take on the 
role of structuring the planning process to 

involve many other partners in thinking about and doing the work. This 
itself is framed in terms of what partners can do to get children healthy 



Cabinet officials in Health 
and Human Services, 
and Education 
Child care providers 
Child care workers 
Children and youth 
City council members 
Civic and neighborhood 
organizations 
Community based 
organizations 
Corporations 
County Supervisors or 
Commissioners 
Doctors 
Educators 
Elders 

Faith community 
Family support centers 
Foundations 



Organizations 
Hospitals and clinics 
Juvenile probation 
Legislature 
Mayors 
Media 

Medical, pediatric and 
obstetric associations 
Parents 

Parks and recreation 
Police 

Public health 
Recipients of service 
School boards 
Sheriffs 
Social Services 
Superintendents 
Tobacco/ substance abuse 
treatment 



will be easier if the work 
and ready for sehool. 



Note: Make sure you eonsider the other efforts in your state or community concerned 
with young children and their families. Many states, for example, have efforts under way 
to improve school readiness. SECCS grantees should not seek to duplicate or supercede 
these efforts. Have an honest and open discussion about how best to do this work 
together. This will involve linking or joining the efforts together in some form. 



Get good staff It is important to have at least one staff person who can help the state grantee 
and its partners organize their work, and support the process of gathering and making sense of 
the input from others. 



12 




Gather ideas from many others. There should be a process by which many partners can offer 
their opinions and experience on the question: “What will it take to make all children in this 

community healthy and ready for school?” Partners should also 
actually do some of the things they suggest (either as part of the 
formal SECCS plan or separately), including no-cost and low-cost 
actions. Methods of including others in this work could include 
public hearings, focus groups, and special committees for different 
parts of the planning process (e.g., data, what works, 
information/research, etc.). The state grantee should also actively 
seek to learn about the successes and failures of those outside the 
state in efforts to improve health and school readiness for young 
children and their families. 

One point worth emphasizing; It is very important to include parents 
and young people from diverse communities as part of this process. 
We often think about “doing for” people. And instead we should think about “doing with.” Other 
countries (e.g., Norway) are committed to learning from their youth, not just teaching them. We 
should do the same. 

Not everyone can be at the table for every meeting. This is a 
republic, not a theoretical democracy. It is important to balance 
processes so that they are truly inclusive, but also manageable. This 
means that lots of small meetings for gathering input and 
performing tasks may be better than just one or two giant ones. On 
the other hand, some jurisdictions have had marked success with 
children’s summits and other large gatherings. Consider all of the 
choices you have about how to involve people in this work. 

Remember: Many different interests will compete to frame “the 
message” of the Initiative. It is important to keep going back to 
basics. SECCS is about the well-being of all children prenatal 
to age five. It is about whether they are healthy and ready for 
school. At this highest level view, there are few who can 
disagree. Whenever the process gets off track, go back to these 
results. 



Indicators help us 
know how we could 
recognize these 
conditions, and 
how we can know if 
we are making 
progress. Without 
indicator data, we 
are left to argue 
about perceptions 
and anecdotes. 



Ask partners 

“What will it take 
to make all children 
in this community 
healthy and ready 
for school? ” not 
“What programs 
should we fund? ” 



2 . Choose indicators 

The plain truth is that it is hard to find good data about the well-being of children prenatal to age 
five. We often don’t count things until children enter school. Data systems for young children 
lag behind data systems for all children, which lag behind data systems used by government 
which lag behind data systems used by business and the private sector. To compound the 
problem, what we count is usually things that have gone wrong: child abuse, child neglect. 
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injury, death, hospitalizations etc. Very rarely do we count positive situations, characteristics or 
events. 

In spite of these problems, it is possible to find 
indicators for children healthy and ready for 
school. It is important first to revisit the 
purpose of choosing indicators. It is to help us 
know how we could recognize these conditions, 
and how we can know if we are making 
progress. Without indicator data, we are left to 
argue about perceptions and anecdotes. If we 
are to be business-like about improving the 
conditions of well-being for these children, 
then we must be business-like about using data 
to steer our decisions and assess our progress. 

Start by assessing experience. How do we 
experience children healthy and ready for 
school? Partners around the table can create a working list of “experiences” in a brainstorming 
session. It is possible to add to this list from consultation with community members, 
professionals, parents and the academic community. By experience, we mean, how do we see, 
hear, or feel the condition? What do we see on the street? What do we see in our everyday work 
and personal lives? Remember that different cultures and communities may experience health 
and school readiness in different ways. 

There are two reasons for starting with experience. First, each experience is a pointer to a 
potential indicator. If we experience children absent from child care or kindergarten due to 
illness, we can possibly count absentee rates in child care or kindergarten. If we experience 
children playing safely on playgrounds, we can possibly count rates of playground injury for 
young children . 

The second reason for starting with experience is that it grounds the work in the common sense 
view of every day citizens. Too often, planning processes are the province of professionals and 
providers who talk in esoteric and inaccessible ways. If this work is to take hold in the 
community and energize the community to take action, it is necessary to build and communicate 
the work in clear and common sense ways. This is not an argument against rigor and discipline. 
Quite the opposite. It is an argument to start the disciplined thinking process where our partners 
and our constituents are. 



Children Healthy and Ready for School 

- Enthusiastic about going to school 

- Dressed appropriately for the season 

- Familiar with letters and numbers 

- Does not experience violence 

- Interacts appropriately with peers 

- Shows social interaction skills on the 
playground 

- Hygienic in bedroom and bathroom 

- Well nourished 

- Coordinated fine and gross motor skills 

- Parental enthusiasm 

- Positive self image 

- Able to communicate 



See the work of the Search Institute (www.search-institute.org) on measuring the positive well-being of young 
people. 
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Finally, the way we experienee results ean be used to drive the thinking and planning proeess 
where indieator data is insuffieient. We may have trouble finding good data to assess whether 
ehildren are well nourished or soeially developed at sehool entry. This does not mean that these 
eonditions are unimportant. We ean think together about “what works” to produee these 
eonditions and use this thinking to fashion our aetion plan.'^ 

The box above gives some ways people might experienee the eonditions “ehildren are healthy 
and ready for sehool.” Your proeess eould take this 
as a starter list and add to it. 

Develop a set of candidate indicators. The box at 
the right gives some starting points for identifying 
eandidate indieators.'^ Partners in the state systems 
building proeess should offer up their ideas as well. 

And there are resourees whieh ean help. The 
Foundation Consortium has developed a guide to 
indieators in California. States and eounties aeross 
the eountry have developed report eards on ehild 
well-being whieh ean illustrate how those outside 
the eounty have seleeted indieators.'^ And 
eommunities may have unique resourees in this area 
if, for example, they have eommissioned surveys of 
families or youth. 

Remember : It is important to inelude as 
many members of the eommunity as possible 
in this thinking proeess. And be sure to tap 
the expertise of your partners in the 
aeademie eommunity, some of whom have 
spent their whole eareers thinking about 
these very questions. 

A word about the notion of leading and lagging indieators. In eeonomies, we have leading and 
lagging indieators of the health of the eeonomy. Leading indieators are indieators whieh show a 
ehange of direetion before the ehange appears in the general eeonomy (e.g., orders for durable 
goods). Lagging indieators refieet the ehange in the eeonomy after it has happened (e.g.. 



Potential Indicators 

- Percent of low birth-weight babies 

- Rate of pre-term deliveries 

- Rate of infant deaths 

- Rate of severe physical and developmental 

complications 

- Rate of lower respiratory tract infections 

in infants and children under 18 months 
of age 

- Rate of hospitalizations 

- Percent fully immunized at age two and five 

- Percent fully ready for school as measured 

on a kindergarten entry assessment 
instrument (See footnote below) 

- Reading at grade level: L‘, 2"^, 3’'^ grades 

- School attendance rate for kindergarten- 

- Percent who pass kindergarten and go on to 

L* grade 

- Peabody verbal - average score or better at 

kindergarten entry 

- Rate of behavior referral among 

kindergarten students 

- Rate of child abuse and neglect ages zero to five 

- Rate of entry into foster care ages zero to five 



See http://www.raguide.org Question 2.9 “What do we do if we don't have any good data at all?” 
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For an example of a full kindergarten entry assessment system, see the Maryland Department of Education’s 

website: http://www.msde.state.md.us/special_reportsanddata/kindergarten_report_2002/index.htm 

1 8 

See “Quality of Life Indicators for Children and Families,” Foundation Consortium, 1998 on 

www.foundationconsortium.org. 
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See “A Guide to Selecting Results and Indicators,” The Finance Project, Atelia I. Melaville, May 1997 
See also http://www.raguide.org/resources_references and click on “Sites with Exemplary Reporting on Results and 
Indicators,” and the new section on “Data Resources” under Index of Tools. 
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unemployment rates). When it comes to the well-being of young children (prenatal to age five) 
much of the data we have are lagging indicators. The percentage of 3'^'* graders reading at grade 
level is a lagging indicator of how ready those children were for school three or four years 
earlier. These are still valuable measures. And it is possible to gear the planning process around 
“What would it take to produce better 5'^^ grade reading scores four years from now? ” Lagging 
indicators bring a healthy and useful perspective. 

Choose the best of what’s available. Given a set of candidate indicators, it is then possible to use 
criteria to select the best indicators to represent the result. Using the best of what’s available 
necessarily means that this will be about approximation and compromise. If we had a thousand 

measures, we could still not fully capture the health and readiness of 
young children. We use data to approximate these conditions and to 
stand as proxies for them. There are three criteria which can be used 
to identify the best measures: 

Communication Power .- Does the indicator communicate to a broad 
range of audiences? It is possible to think of this in terms of the 
public square test. If you had to stand in a public square and explain 
to your neighbors "what we mean, in this community, by children 
healthy and ready for school," what two or three pieces of data 
would you use? Obviously you could bring a thick report to the 
square and begin a long recitation, but the crowd would thin quickly. 
It is hard for people to listen to, absorb or understand more than a 
few pieces of data at a time. They must be common sense, and 
compelling, not arcane and bureaucratic. Communication power 
means that the data must have clarity with diverse audiences. 



Using the best data 
available 
necessarily means 
that this will be 
about 

approximation and 
compromise. If we 

had a thousand 
measures, we could 
still not fully 
capture the health 
and readiness of 
young children. 



Proxy Power .- Does the indicator say something of central importance about the result? (Or is it 
peripheral?) Can this measure stand as a proxy for the plain English statement of well-being? 
What pieces of data really get at the heart of the matter? 

Another simple truth about indicators is that they run in herds. If one indicator is going in the 
right direction, often others are as well. You do not need 20 indicators telling you the same thing. 
Pick the indicators which have the greatest proxy power, i.e. those which are most likely to 
match the direction of the other indicators in the herd. 

Data Power .- Do we have quality data on a timely basis? We need data which is reliable and 
consistent. And we need timely data so we can see progress - or the lack thereof - on a regular 
and frequent basis. Problems with data availability, quality or timeliness can be addressed as part 
of the data development agenda. 



Identify primary and secondary indicators, and a data development agenda. When you have 



16 





assessed the candidate indicators using these criteria, you will have sorted indicators into three 
categories: 

Primary indicators : those three or four most important measures which can be used as proxies in 
the public process for children healthy and ready for school. You could use 20 or 40, but 
peoples’ eyes would glaze over. We need a handful of measures to tell us how we’re doing at the 
highest level. 

Secondary indicators : All the other data that’s any good. We will use these measures in 
assessing the story behind the baselines, and in the “behind the scenes” planning work. We do 
not throw away good data. We need every bit of information we can get our hands on to do this 
work well. 

A data development agenda : It is essential that we include investments in new and better data as 
an active part of our work. This means the creation of a data development agenda - a set of 
priorities of where we need to get better. 

It is a judgment call about how much to spend on such an agenda. Spending for data or any other 

administrative function should always be carefully balanced with spending which directly 

benefits children and their families. As a general rule administrative spending in health and 

social service systems should not exceed five to 10 percent of total spending . And data 

investments are only part of that amount. Since SECCS funds are very limited, this means that 

other partners will have to support this effort. And it means that not all data has to be of the 

highest research quality. At this stage of our learning about how to use data to make decisions, it 

is OK to use sampling and other techniques to get usable information that may not meet strict 

2 1 

academic research standards. 

3, Develop Baselines and the Story Behind the Baselines 

Baselines tell us where we’ve been and where we’re headed on the most important measures. 

The story behind the baselines helps us understand the causes and forces at work, so that we can 
take action to change for the better. 

Develop baselines for each of the indicators The purpose of baselines is to show over a multi- 
year period where we have been and where we are headed on each indicator. Baselines have two 
parts: an historical part which shows where we’ve been, and a forecast component that shows 



20 

You want to be able to say “90 to 95 percent” of spending goes to directly serving children and families.” 

21 

For example, in prior work in Maryland, when we needed data on what was happening in the caseload, we would 
sometimes send a staff person to pull and read 20 or 30 cases. It was not scientific, but it was fast and gave us a 
handle on the immediate policy problem we were trying to solve. 
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where we’re heading if we stay on our current course. 



Forecasting is an art, not a science. The best forecasting is not about 
technical statistical analysis. It involves people who know “what’s 
happening on the street” and who can create two or three believable 
scenarios of the likely future. Statistical folks can help this process by 
analyzing trends and presenting data. But do not relinquish control of 
the forecasting work to the statisticians. Forecasts should reflect the 
consensus view of key partners about where we are heading. 

Baselines serve two purposes. First they allow us to ask and answer 
the question: “Is this future OK?” If the percentage of 3“^ graders 
reading at grade level has been declining for several years, “is it OK 
for this decline to continue?” “Look where we’ll be in two years if 
this continues!” Most processes of serious change start with the 
members of the community saying “This is NOT OK. We can do 
better.” Baselines with forecasts allow you to have this discussion. 

Second, baselines allow us to assess 
progress in terms of doing better than 
the baseline. This allows us to “count” 
as progress when we have slowed the 
rate at which things are getting worse, 
before we fully turn around and go in the 
right direction. For example, a school, 
where reading scores have been 
declining for the past 10 years, will have 
made significant progress if it can hold 
scores even for one year. This stands in 
contrast to the usual definition of 
success: that things get better right 
away. This is often unrealistic when 
trends have been headed in the wrong 
direction for a long time. It takes time to 
turn the curve on such a trend line. If we 



Kindergartners' Immunizations 
Up-To-Date at Age 2 




1996 1997 1998 1999 2000 2001 2002 

Source: California Department of Health Services, immunization Branch 
Prepared by Michael Padding for FPSI 



Baselines allow us 
to ask and answer 
the question: “Is 
this future OK? ” 
Most processes of 
serious change 
start with the 
members of the 
community saying 
“This is NOT OK. 
We can do better. ” 



The best way to calculate “reading at grade level” is in relation to a level of achievement that all children can 
strive to attain. The numbers then tell us what percentage of children achieve at or above that standard. Often, 
scores are presented based on state or national “norms” which means, by definition, that 50 percent of children will 
be above, and 50 percent will be below, the norm. 
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do not use baselines to measure success, we set ourselves up for failure by creating unrealistic 
expectations for quick fixes. The box right displays the history part of a sample baseline 
prepared for training sessions for California First 5 commissioners. 

Tell the story behind each baseline, or the story for all the baselines together. What are the 
causes and forces at work? Why does this picture look the way it does? Why are only 67 percent 
of our children immunized at age two? Why are so few of our 3’^‘* graders reading at grade level? 
Why is it getting worse or improving so slowly? This is the “epidemiology” part of the work. 
Each part of the story is a pointer to action in the next stage. If one of the reasons 3’^‘* grade 
reading scores are going down is that parents don’t do a good job helping children build reading 
skills, then this is a pointer to parent education, support for library reading programs, campaigns 
to get reading material into the homes of families with young children, etc. Be prepared for 
different stories and earnest debate. But it is not necessary to reconcile different perspectives and 
agree on a single story. As different partners add their perspectives, a rich picture will emerge 
about what is happening in the community and why children are, or are not, healthy and ready 
for school. 

Remember : When considering the story behind the baselines, make sure, again to involve 
as many partners from as many different and diverse communities as possible. It is 
particularly important to involve parents in this thinking process. The “people we are 
trying to help” often know more about causes and likely successful approaches than the 
professionals. 

In doing this work, it will be useful over time to develop the indicator baselines and the story 
behind the baselines into a periodic report card on “children healthy and ready for school.” 
Making this report card part of a larger report card on child and family well-being will help link 
the work of the grantee and its partners to other efforts in the state to improve results for children 

23 

and families. 

Another type of story which needs to be told is the cost of bad results. What is the price we pay 
when children are not healthy and not ready for school? Such costs show up in many places: as 
remedial education costs in the school system, as health care costs in the public and private 
sector, and later in such things as teen pregnancy and juvenile crime. Showing the costs of bad 

24 

results helps make the economic case for additional investments in children and their families. 

4, Consider what works - What would it take? 

• Given the results we want (children healthy and ready for school) ; 

• Given the indicators of those results and the story behind the indicator baselines; 

• Given the partners around the table 



See http://www.raguide.org/resources_references and click on “Sites with Exemplary Reporting on Results and 

Indicators.” 
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See “A Guide to Developing and Using Family and Children’s Budgets” pages 36, 37 for more information on the 
cost of bad results, accessible on http://www.resultsaccountability.com 
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What works, what could work, to turn these eonditions around? What would it take to have all 
children prenatal to age five healthy and ready for school?^^ The answers are a eombination 
of seienee and eornmon sense, a eombination of ehanges in publie and private serviee systems 
and aetions of parents, businesses, the media and many other partners. 

Look at the research. The seienee part is 
about the researeh that has been done over 
the last 20 plus years on programs that 
aetually make a differenee in the well- 
being of young children and their families. 

While some of this is still eontroversial, we 
know that quality child care, regular health 
care, family support and parent education 
all play a key role in the healthy 
development of ehildren. The chart 
displays the references in California’s First 
5 legislation to what works. SECCS states 
eould use these eategories as a starting 
point for eonsidering what works. There 
are some references in the appendiees to 
other important “what works” resources, 
including the recent work of the Annie E. 

Casey Eoundation on its Pathways 
Project.^^ 

Don V be limited by research. The eornmon sense part: not all things that work have been proven 
by the researeh eommunity. What works ideas should build on your experienee and what you 
and your partners know about the state and the eommunities in whieh you live. What do you 
think would work herel Involve as many partners as you ean in this thinking proeess, ineluding 
the parents of young children. Be disciplined about this part of the work as well. Not all ideas are 
good ideas. Test eaeh idea against the questions: “Why do you think this would work? Would 
this make a differenee in the well-being of young ehildren and their families? Would it help turn 
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the eurves we are trying to turn?” 



Some of What Works Ideas 
Refereneed in California’s Eirst 5 Eegislation 

1. Comprehensive, collaborative, integrated . 
consumer oriented, easily accessible system of 
information and services 

2. Child care : high quality, accessible, affordable 

3 . Parent education : nutrition, care and nurturing, 
child development, child abuse prevention 

4. Child health services : prevention, diagnosis, 
treatment, pre and post natal maternal health, 
nutrition, tobacco and substance abuse treatment 

5. Parental support : family support centers, 
domestic violence prevention and treatment, family 
services and counseling 

6. Mass media communications : for the general 
public on nurturing, parenting, selection of child 
care, prevention of tobacco and drug use by 
pregnant women. 



25 

The question, “What would it take?” is emerging as the most important question we need to ask ourselves. It is far 

more powerful than “What could we do?” or “What would help?” 

26 

The “what works” section of this paper is actually shorter than other sections even though this is the heart of the 
work. That is because the paper is intended to provide a framework for answering what works questions, not the 

answers themselves. For links to other what works resources, seewww.raguide.org/resources_references.htm 
27 

This is where theory of change or logic model thinking can be most valuable. The answer to the question “Why 
do you think this will work?” requires the articulation of a theory of change, or a cause and effect progression which 
leads to the desired effect. It can be used to test our ideas about what works. 
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Consider what has worked in other places: JhQXQ is a growing body of experience from other 
counties, states and countries about what works to improve the well-being of children and 
families. This is sometimes referred to as “best” or “promising” practice. And a number of 
books, journals and websites provide access to this experience. Some of these are listed in 
Appendix A. Localities with successful efforts are usually willing to host visitors, and this can be 
a powerful way to get beneath the surface of advertised claims about what really worked or 

didn’t work. Some technical assistance centers (also listed in 
Appendix A) can help arrange site visits or “peer to peer” 
consulting. It goes without saying that what works in one 
community may not work in another. So, look for experience in 
states, counties and communities with economic and demographic 
characteristics similar to your own. 

Consider no-cost and low-cost ideas. No-cost and low-cost ideas 
can be among the most powerful parts of your plan, and are 
particularly important in this time of shrinking budgets. We have a 
tendency to think about everything as a money problem. And while 
money is certainly important, it is not the only way to turn a curve. 
There are many ways for partners to make contributions to this work 
(e.g., use of volunteers, advertising by the media, family friendly 
policies by the business community, support groups by the faith community, streamlined policy 
or procedure by public agencies etc.) that make a crucial contribution at low cost and without 
using public funding sources. When groups are given the challenge 
to turn a curve (like reading scores or immunization rates), and are 
asked to include at least one no-cost or low-cost idea, it often 
happens that half to two thirds of the good ideas are no-cost or low- 
cost. The simple act of asking for no-cost and low-cost ideas has the 
effect of giving people permission to think differently. Consider the 
following simple rule: Any plan which does not contain a significant 
number of no-cost or low-cost action items is not complete. 



Use pointers to action. There are two pointers to what works in the 
preceding steps of the process. Each element of the story behind the baseline is a pointer to 
action. And each partner or potential partner is a pointer to action. State grantees should seek 
advice from a wide range of partners on what it would take to get all children healthy and ready 
for school, and should ask each partner the following questions: 

. What is your best assessment of whether our children are now healthy and ready for 
school and why? 

. What could work in this state, this county, this community, to improve this situation 
(including no-cost and low-cost ideas)? 



Any plan which 
does not contain a 
significant number 
of no-cost or low- 
cost action items is 
not complete. 



The answer to 
“What works? ” is a 
combination of 
science and 
common sense: 
what the research 
tells us and what 
our experience tells 
us would work here 
in this community. 
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See “funding the plan” in section 5 below. 
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. What can you contribute (time, money and expertise)? 

5, Craft what works ideas into a coherent strategy 

The kind of proeess deseribed above ean sometimes produee a long list of everything anybody 
ever thought was a good idea to do for children and families, completely undiseiplined and 
unaffordable. The triek in this work is not to ereate such a laundry list, but a coherent strategy, 
that we ean aetually afford to implement that will aetually produee the results we want. 

Assess your ideas against criteria. One way to do this is to assess the “what works” ideas 
against established eriteria. Four eriteria are offered for your eonsideration: 

Speeificity : Is the proposal speeific about what will be done, when and by whom; or is it a 
rhetorical statement of need like “end poverty and eure disease.” Proposals need to take the form 
of an aetionable item whieh ean be funded and implemented. 

Leverage : How great an impaet will this proposal have on the 
eonditions we are trying to ereate, on the curves we are trying to 
turn? We are looking for notions whieh are high leverage, not token 
efforts. 

Values : Is the proposal oonsistent with our personal and our 
oommunity’s values? There are many proposals whieh are 
potentially effeotive which violate important prinoiples of equity and 
fairness. The best approaohes must be true to oommunity values and 
must take into aooount differences in cultures and community norms. 

Reach : Is it feasible and affordable? Can it be done this year, next 
year, or over three to 10 years. This eriteria ean help space out our 
efforts over time. 

We are looking for notions whieh are high on all four eriteria: notions whieh are speoifio, high 
leverage, oonsistent with our values and whieh ean be implemented sooner rather than later. 

Eaoh proposal ean be rated high, medium or low on these four eriteria. Those that rise to the top 
ean beoome the first year’s plan. Others that are high on the first three eriteria, but lower on the 
fourth ean be targeted for later years. It should be expected that the first year’s plan will inolude 
many no-oost and low-oost notions, sinoe these will be very high on feasibility and affordability. 

Fit the pieces together. Having seleoted priorities for notion is not the same as having a ooherent 
plan. We need to oonsider how these pieoes fit together in a system of services and supports, not 
just a loose confederation of good ideas. This means consideration of the need to create a 
comprehensive, collaborative, integrated, consumer oriented and easily accessible system of 



The trick in this 
work is not to 
create a laundry 
list, but a coherent 
strategy, that we 
can actually afford 
to implement that 
will actually 
produce the results 
we want. 
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services and supports for young children and their families. 

SECCS Initiatives should ereate a special part of the proeess (a subeommittee or task force) to 
look at the following questions: 

• How is the system of services currently eonfigured? 

. What parts of the system are diffieult for families with young children to aceess or 
negotiate? 

• How can services be made more aeeessible to families of different eultures? 

. Where do we have opportunities to break down walls between serviee systems and lessen 
duplication and bureaucracy? 

This is a major purpose of the SECCS grant and should be a major focus of attention for SECCS 
partners. This group may identify additional action items whieh require funding, sueh as the 
creation of a resouree and referral network for child care, the placement of new sereening and 
diagnostic services in family centers, or 
the addition of evening and weekend 
hours for health care or ehild eare 
services. These can be added to the 
“what works” agenda and ranked 
against other proposals. 

Many of the ehanges neeessary to 
improve the service system will involve 
no-cost and low-cost actions such as the 
loeation of services in schools, creation 
of common forms across systems, 
shared intake and assessment services, 
or wrap around funding for children in 
out of home care. 

One eommunity labeled their work to 
make the system aeeessible “No wrong 
door” meaning that every point of 
eontact with the parents and children 
should involve knowledgeable workers 
who could help access any service. 

Another image which might be helpful 
is that of a service system with a “front 
room” and a “back room.” In the front 
room, families and children get what 
they need in a seamless, culturally 
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Wording from California’s First 5 law. 



A Possible 
SECCS Plan Outline 

A. What’s at stake? 

1 . The importance of positive development for children 

2. The cost of bad results if we fail 

B. How are children prenatal to age five doing? 

1 . The conditions of well-being (results) 

we want for young children and their families 
(in plain English, plain Spanish, etc.) 

2. How we recognize these conditions in 
our day to day experience 

3. How we measure these conditions: indicators 
of well-being 

4. Where we’ve been; where we’re headed: 
indicator baselines and the story behind the 
baselines 

C. What works - What will it take to do better? 

1 . Partners who have a role to play 

2. What’s worked in other places; What we 
think will work here (best practices, best 
hunches, and no-cost low-cost ideas) 

3. How we will create a comprehensive, 
integrated, consumer oriented, easily 
accessible system of services 

D. What we and our partners propose to do! 

1 . This year 

2. Next year 

3. Three to 10 years 

-FPSI 
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competent and eonsumer friendly way. In the back room we run the financial and technical 
systems neeessary to make the front room work. 

The product of this work should be a visual map of how the service system now looks, and how 
it should look, from the consumer’s point of view. This can be used as a tool to move the system 
toward becoming more friendly to families with young ehildren. 

A word about “systems change” : Systems ehange is one of the foremost stated purposes 
of the SECCS grant, and you might have thought that the whole paper would be about 
that. But change in formal services systems is a neeessary but not sufficient condition for 
improving the well-being of young ehildren and their families. It is actually possible for 
serviee systems to improve while conditions for children and families get worse. We 
should not confuse improving services for improving results. So, systems ehange can and 
should be a central part, but only a part, of SECCS plans. 

Remember : The purpose of this work is not planning. The purpose of this work is doing. 
There is a tendency for planners to become so enamored of their planning processes that 
they forget there are other things to do in addition to planning. It is necessary to do the 
best planning possible without letting the planning process itself become the point of the 
work. The point of the work is not planning, but action to improve results. 

Fund the plan. A crucial part of developing any action plan is consideration of how the elements 
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can be funded. It is beyond the scope of this paper to fully explore this part of the work. 

SECCS grants represent only a small fraction of the resources needed to make a differenee in the 
well-being of young children and their families. And cuts in federal, state and local funding for 
services make this ehallenge even more difficult. SECCS and Maternal and Child Health (MCH) 
dollars are scarce resources and should be used as much as possible to leverage actions and 
investments by other partners. The business of leveraging other resources may be thought about 
in terms of the following priorities: 

. Eirst, no-eost or low-eost items 

• Next, items fully funded by other partners without MCH or SECCS dollars 
. Next, items funded jointly with MCH or SECCS and other funds, and 
. Einally, items funded solely with MCH or SECCS dollars 

The state systems building effort should try to get the most effect for the fewest dollars by 
minimizing items in the last category. 

It is important to eonsider the ways in whieh non-federal funds ean be used as mateh for open- 
ended federal funding under Medicaid (Title XIX) and Eederal Eoster Care (Title IVE). These 



30 

A number of excellent publications, as well as technical assistance, are available from the Finance Project to help 
think about how to fund services for children and Families (http://www.financeproject.org). See also “The 
Cosmology of Financing” on http://www.resultsaccoutability.com under “Papers you can read on line.” 
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sources can sometimes be used to help pay for services with a medical or therapeutic component, 
or whieh address the needs of children at risk of abuse or negleet. Don’t let the availability of 
sueh a match determine what you decide to do. But for things you want to do anyway, this ean 
have the effeet of multiplying the resources. This opportunity to match non-federal funding 
makes it even more important to attract resources to this effort beyond SECCS and MCH funds. 

In doing this work it may also be useful to develop a map of how funds are now deployed for 
services to young children and their families. A growing number of places in the country have 
developed Family and Children’s budgets to provide sueh a picture. The grantee and its 
partners may wish to supplement these efforts by developing an analysis specific to services for 
ehildren prenatal to age five. 

6. Implement the strategy 

You will need an aetion plan whieh says who’s going to do what by when. Typically such plans 

-5-5 

include “goals,” “objeetives” and timetables. Goals are speeific accomplishments whieh are 
planned for the future, such as the creation of a new child care center or the eo-loeation of 
services in one building. Objeetives are the speeifie actions necessary to make this 
accomplishment happen (e.g., securing funding, agreeing on space, designing common forms, 
etc). The timetable tells when each objective should be started and eompleted in order to reach 
the time targeted for the particular accomplishment/goal. 

There are many possible ways to eonstruct and format sueh an action plan. The main thing to 
remember is to keep it simple, and to make it useful. This means avoiding the thousand page 
version that is a bookshelf doeument and nothing else. The bottom line is for the partnership to 
find ways to track the work and determine if it is actually getting done in a timely way. This is 
separate from the question of what work should be done. And it is also separate from the 
performance measures for individual programs (discussed in the next seetion) whieh tell whether 
that partieular program is working properly or not. 



7. Using performance accountability to achieve high quality performance of programs, 
agencies and service systems” 

All performance measures fit into one of four categories, derived from the intersection of 
quantity and quality vs. effort and effect. And not all performance measures are of equal 
importance. The most important measures tell us whether our clients or customers are better off 
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This is a real risk. There is a long history of chasing money, which means we fund the things that generate dollars, 
not necessarily the things that most need to be done. See “The Cosmology of Financing” and the Policy Brief 

“Reforming Finance, Financing Reform” referenced in Appendix A. 
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Notable Family and Children’s Budgets have been produced by Los Angeles County, Contra Costa, San Francisco 
and Sonoma counties in California. Go to http://www.resultsaccountability.com/links.htm for information on other 

leading sites. 
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Choose different words to label these ideas if you like, using Appendix B. 
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as a consequence of receiving the service (Is 
Anyone Better Off: lower right quadrant). 

We call these measures “client or customer 
results”^"^ These are measures which gauge 
the effect of the service on peoples lives. 

Usually, in programs which directly deliver 
services to people, client results have to do 
with four dimensions of “better-offness.” 

Skills & Knowledge, Attitude, Behavior 
and Circumstance. Did their skills or 
knowledge improve; did their attitude change 
for the better, did their behavior change for 
the better, is their life circumstance improved 
in some demonstrable way? So, for example, 
if you are overseeing a child care program, 
you might want to measure such things as the 
percent of children with basic literacy skills 
(skills), the percent of children with a 
positive self image (attitude); the percent of 
children exhibiting disruptive behavior 
(behavior) and the percent of children who 
are up to date on their immunizations, and the percent who go on to succeed in U* grade 
(circumstance). 

The second most important measures are those that tell whether the service and its related 
functions are performed well (How well did we do it? upper right quadrant). These measures 
include such things as timeliness and accessibility of service, cultural competence, turnover rate 
and morale of staff. These measures are used by managers to steer the administration of the 
program. If things are late, they work to make them timely. If turnover is high, they work to 
retain staff. 

Don’t accept lack of control as an excuse^ Now the first thing you’re going to say is “Wait a 
minute. What does child care have to do with whether or not children are up to date on 
immunizations? This is a good example of a performance measure where child care has very 
little control over whether the circumstance improves. Child care can make a contribution to the 
immunization status of its clients. Quality child care can help parents and children understand 
the importance of regular preventive health care and can help parents understand and access the 
health care system. But child care by itself can not do these things. So isn’t it unfair to track 
immunization rates for children in care? 



Types of Performance Measures 



Quantity 


Quality 


How Much Did We Do? 


How Well We Do it? 


# CustQmers served 

(by diaracteristic) 


% Common measures 

(e.g. staff turnover rate) 


# Activities 

(by type ofactivity) 


% Activity specific measures 
(e.g. activity timeliness 
or accuracy) 


Is Anvone Better Off? 


^ with Improvement in: 


% with Improvement in: 


Skills / Knowledge 


Skills / Knowledge 


Attitude 


Attitude 


Behavior 


Behavior 


Circumstance 


Circumstance 

i 



Most Important 
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Or client, patient, or student results (or outcomes) depending on what system you live in and what words you 
choose to use. 
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If you look at the other measures listed for ehild care (literacy skills, self image, disruptive 
behavior, first grade success) you will notice that these measures are also beyond the capacity of 
the child care provider to completely control. 

And that’s the point: all programs performance 
measures are affected by many factors beyond 
the control of the program. This lack of control 
is usually used as an excuse for not doing 
performance measurement at all. Turnover rate, 
staff morale, you name it is “beyond my 
control.” 

In fact, the more important the performance 
measure (e.g., children successful in grade) 
the less control the program has over it. This is 
a paradox at the heart of doing performance 
measurement well. If control were the 
overriding criteria for performance measures 
then there would be no performance measures at 
all. The first thing that we must do in 
performance measurement is get past the control 
excuse, and accept that we must use measures 
we do not completely control. 

Create a performance accountability system 
useful to managers, one that takes this control 
paradox into account. We do this in three ways. 



First, we ask managers to assess their performance on these measures 
- not on the basis of some absolute standard - or on how other 
providers are doing - but on whether they are doing better than their 
own history. We do this using the same technique used for cross 
community indicators: the notion of baseline. For each performance 
measure we ask managers to present a baseline of the history of their 
program’s performance, and where their performance is headed. We 
ask them to do better than their own baseline. 

This is the central way in which businesses use data. How are we 
doing compared to our own history. Later when you have the 
sophistication and the data, you can begin to develop and use 
comparisons to the performance of other similar providers with 
similar mixes of easy and hard cases. And later still, we can compare 
to standards, when we know what good performance looks like. 



Note : In some services, like child care, we have progressed to the 



Not all 
performance 
measures are of 
equal importance. 
The most important 
measures tell us 
whether our clients 
or customers are 
better off. We call 
these measures 
“client or customer 
results 



Performance Accountability 
for Programs Agencies and Service Systems 

1 . Who are our customers, clients, people we serve? (e.g 
children in a child care program) 

2. How can we measure if our customers/clients are better off? 
(performance measures about client results (e.g., percent of 
children with good literacy skills) 

3. How can we measure if we are delivering service well? (e.g., 
child staff ratio, turnover rate, etc.) 

4. How are we doing on the most important of these measures? 
Where have we been; where are we headed? (baselines and the 
story behind the baselines) 

5. Who are the partners who have a potential role to play in 
doing better? 

6. What works , what would it take, to do better than baseline? 
(best practices, best hunches, including partners’ contributions) 

7. What do we propose to do? (multi-year action plan and 
budget, including no-cost and low-cost items) 

-FPSI 
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point where we have standards for performance measures about how well we delivered 
service. In child care we know what quality service delivery looks like. We have 
standards for staffing ratios, percent of staff with defined qualifications, timeliness of 
service, safety etc. Standards about “Is anyone better off? are much more difficult to 
establish, and meaningful standards for this type of measure are rare. 

Next we ask managers to think about the partners who have a role to play in doing better. 
Programs cannot produce the most important results for customers by themselves. And, finally 
managers must ask and answer: “What works to improve performance?” Out of this process we 
ask managers to present their best thinking about what needs to be done. This thinking process 
is summarized in the seven questions in the sidebar chart. These seven questions should be asked 
and answered at every intersection between a supervisor and a subordinate throughout the 
system. This means that the questions should be used in supervisory conferences between 
supervisors and those who work for them. 

Performance measurement and accountability operates, not just at the program level, but also at 
the agency and service system levels. The same thinking process is used at these higher levels of 
accountability: Who are our customers? How could we measure if our customers are better off? 
How could we measure if we’re delivering service well? How are we doing on the most 
important of these measures? Who are the partners who have a role to play in doing better? What 
works to do better? And what do we propose to do? Asking these questions at the system level 
requires that we think about families and children who are common customers of different 
agencies. And it requires that we think about performance measures about how the system as a 
whole functions. Examples of such measures include: 

How well did we do it? measures : Average number of caseworkers or case managers per 
family, average number of different offices a typical family has to visit each month, 
average time, distance, or number of bus changes it takes to get from home to the 
provider’s office, number and percent of all the state’s children in residential placement 
who are placed out of state. 

Is anyone better off? measures : Percent of all children in the service system with health 
insurance and a medical home, percent of children in the service system fully up to date 
on immunizations, percent of parents served by the system who are employed. 

Since no single manager is responsible for the service system as a whole, this kind of 
accountability requires special structures, like Children’s Cabinets or Family and Children’s 
Collaboratives which can track system performance and work together on making 
improvements. The SECCS grant can be a catalyst for creating this kind of structure and 
encouraging this kind of thinking. 

8. Play it again. 

The planning process described above should be followed by an equally important process to 
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track progress. One way to strueture this is to sehedule quarterly review sessions to assess 
progress on implementation of the plan, and to revisit eaeh key step in the thinking proeess 
(baselines, story, partners, what works) to keep making the aetion plan better. Eaeh time you 
iterate this proeess, the aetion plan gets better. 



“In the past 10 
years the eounty 
has made dramatie 
progress in the 
well-being of its 
youngest ehildren 
and we are 
beginning to see 
these improvements 
pay off in the well- 
being of 
adoleseents and 
young adults. ” 

The entire planning 
proeess ean be 
thought of in terms 
of “What would it 
take to be able to 
have a press 
eonferenee like this 
one in five or 10 
years?” 



An imaginary press conference in 10 years: What we would like to 
be able to say... 

“Good afternoon Ladies and Gentlemen. We are here today to 
present the results of our 10 year campaign to assure that every child 
in this state enters school healthy and ready to learn. The charts in 
your package show that when we started this work, less than half the 
children in the state passed the kindergarten entry assessment, and 
the percent of children reading at grade level was no better than 45 
percent in the W 2 ”“^ and 5 '^^ grades. Less than 60 percent of our 
children were fully immunized at ages two and five, and rates of 
hospitalization, unintentional injury and foster care entry for very 
young children was far above national averages. What ’s worse, 
nearly all of these measures were headed in the wrong direction 

In the past 10 years the state has made dramatic progress in the well- 
being of its youngest children and we are beginning to see these 
improvements pay ojf in the well-being of adolescents and young 
adults. Today, nearly all children (96%) pass the kindergarten entry 
test, and nearly all children are reading at or above grade level in T* 
2 ""^ and 5 '^'^ grades. The health of these children is much improved. 
Nearly every child (99%) is up to date on immunizations at ages two 
and five. Rates of unintended injury and hospitalization have declined 
significantly. And we now are well below the national averages for 
rates of child abuse and neglect and entry into foster care. 

We believe that these improvements are directly related to the 
investments we have made over the past 10 years, using Maternal 
and Child Health and other funds, and also the combined resources 
of time, energy and commitment of many public and private partners 



throughout the county. Some of the most important of these investments include... 



Finally, we believe we are seeing a real and direct financial benefit as a result of these 
improving conditions. Spending for remedial education has declined, foster care costs are 
significantly lower, and we are beginning to see significant reductions in our juvenile justice 
caseload. Health care costs for young children have shown a marked shift away from expensive 
remedial care and toward the higher regular utilization of preventive and well-child care. And 
the state was recently rated among the best places in the country to raise children, a fact we 
know has affected the decisions of several businesses to stay in or relocate to our communities. 
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Your packages include many of the details behind these remarkable accomplishments, and list 
the many partners who have made contributions over this period. We would like to express our 
deep gratitude to these people and organizations. We would now be glad to answer any 
questions you may have. ” 

The entire planning proeess ean be thought of in terms of “What would it take to be able to have 
a press eonferenee like this one in five or 10 years?” Answer this question, and you ean throw 
the rest of this paper away. 



Conclusion 

We have an historie opportunity to make investments in ehild and family well-being that will pay 
off for deeades to eome. If the proeesses we ereate are all talk and no aetion token efforts, we 
will not be remembered for using this opportunity well. If instead we bring business-like 
diseipline to the demands of improving eonditions of well-being for young ehildren and their 
families, then we have a ehanee to be remembered differently. The approaeh offered in this 
planning guide ean be used to strueture the planning proeess, and future iterations of the 
planning proeess, to produee the legaey of results we want for young ehildren and their families. 
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Appendix A 



Resources 



This appendix provides a list of selected organizations and materials (paper, video and web-based) which may be 
useful in understanding or implementing results and performance accountability. 



Organizations 

> UCLA Center for Healthier Children, Families 
and Commnnities 

10945 Le Conte Avenue 
Ueberroth Building, Ste 1401 
Los Angeles, CA 90095-6939 
310-825-8042 

> Association of Maternal and Child Health 
Programs 

1220 19* Street, NW, Suite 801 
Washington, DC 20036 
202-775-0436 

> Health Systems Research, Inc. 

1200 18* St., NW Suite 700 
Washington, DC 20036 

http://www.hsrnet.com 

> The Fiscal Policy Stndies Institnte 

7 Avenida Vista Grande #140 
Santa Fe, New Mexico 87508 
http://www.resnltsacconntahility.com 
http ://www.ragnide.org 

> The Annie E. Casey Fonndation 

701 St. Paul Street 
Baltimore, Maryland 20202 
410-547-6600 
http://www.aecf.org 
http://www.aecf.org/pathways 

> The Foundation Consortium 

2295 Gateway Oaks Drive Suite 100 

Sacramento California 95833 

916-646-3646 

http ://www .wwlc.org 

http://www.promisingpractices.net 



Publications and other materials 

(Note: The papers marked “ *w ” below can be read on line at the 
website of the publishing organization) 

> http://www.raguide.org The Results and 
Performance Accountahility Implementation 
Guide, created by the Fiscal Policy Studies Institute 
with support from 

The Foundation Consortium, The Colorado 
Foundation, The Annie E. Casey Foundation, The 
Finance Project, and the Nebraska Children and 
Families Foundation. 

> Building a Results Accountahility Framework: 
Video of the July 19, 1999 California 
teleconference presentation for First 5 
Commissioners, sponsored by the California 
Children and Families First State Commission, The 
California Endowment and the Foundation 
Consortium. Copies available at cost from The 
Foundation Consortium (Video of the framework 
presented in this paper) 

> A Guide to Developing and Using Family and 
Children’s Budgets, The Finance Project, Mark 
Friedman and Anna Danegger, August 1998 *w. 

> A Guide to Developing and Using Performance 
Measures in Results-hased Budgeting, The Finance 
Project, Mark Friedman, May 1997 *w. 

> A Guide to Selecting Results and Indicators, 

The Finance Project, Atelia I. Melaville, May 1997 
*w. 

> Pathways to Outcomes: Lizbeth Schorr, Annie E. 
Casey Foundation, http://www.aecf.or/pathways 

> Quality of Life Indicators for Children and 
Families, The What Works Learning Community, 
Foundation Consortium, 1998 



> A Strategy Map for Results-hased Budgeting: 




> Center for Collaboration for Children 

CSU Fullerton EC-324 
800 N. State College Blvd. 

Fullerton, California 92634 
714-278-2166 

> UCSF Child Services Research Group 

44 Montgomery Street, Suite 1450 
San Francisco, California 94104 
415-502-6174 

> The Finance Project 

1000 Vermont Avenue NW 
Washington DC 20005 
202-628-4200 

http ://www.financeproj ect.org 

> The Promising Practices Network 

Sponsored by the Foundation Consortium, the 
Colorado Foundation for Families and 
Children, the Missouri Family Investment 
Trust and the Georgia Academy 

http://www.promisingpractices.net 

> Center for Best Practices 

National Governor’s Association 
http://www.nga.org/CBP/center.asp 

> The Center for the Study of Social Policy 

1250 Eye St. NW Suite 503 
Washington, DC 20005 
202-371-1565 

http://www.cssp.org 

> The California Children and Families First State 
Commission 

501 J. Street Suite 530 
Sacramento, California 95814 
916-323-0056 

http ://www.ccfc.ca.gov 

> The Search Institute 

700 South 3"“^ Street, Suite 210 
Minneapolis, Minnesota 55415 
612-376-8955 

http ://www.search-institute.org 



Moving from Theory to Practice, The Finance 
Project, Mark Friedman, September 1996 *w. 



> The Cosmology of Financing: Financing 
Reform of Family and Children’s Services: An 
Approach to the Systematic Consideration of 
Financing Options, The Center for the Study of 
Social Policy, Mark Friedman, June 1994. 

> Capturing Cash for Kids: A Workbook for 
Reinvesting in Community Based Prevention 
Approaches for Children and Families, The 
Comprehensive Integrated Services Reinvestment 
Project of the Foundation Consortium, Marty Giffin, 
Abram Rosenblatt, Nancy Mills and Mark Friedman, 
September 1998. 

> The Future of Our Children: Long-term 
Outcomes of Early Childhood Programs, Center 
for the future of Children, The David and Lucile 
Packard Foundation, Volume 5, Number 3, Winter 
1995. http ://www.futureofchildren.org 

> Successful Early Childhood Interventions, 

Victoria Hendrick, Susan Neufeld, Melissa 
Del’Homme, & The Consortium for Successful Early 
Childhood Interventions, Los Angeles Department of 
Mental Health, 1999 

> New Approaches to Evaluating Community 
Initiatives, Edited by Karen Fulbright- Anderson, 
Anne C. Kubisch, and James P. Connell, The Aspen 
Institute, 1998. 

> Reforming Finance, Financing Reform for 
Family and Children’s Services, What Works 
Policy Brief, The Foundation Consortium, Mark 
Friedman, January, 2000. 

> Yale Bush Center 

310 Prospect Street 

New Haven, Connecticut 065 1 1 

203-432-9935 



> The Urban Institute 

2100 M Street NW 
Washington, DC 20037 
202-833-7200 

http ://www.urban.org 
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FPSI: 11/10/02 A TOOL FOR CHOOSING A COMMON LANGUAGE 

(and constructing a meaningful glossary) 





Choices 




Chosen Word or Phrase 


Framework Idea 


Common Labels for each Idea 

(Each line represents a 
separate choice.) 


Motifiers - if you must 

{and some notes) 


PICK ONEII 

(Each word or phrase can 
be used only once.) 


A. The Basics 








1 . A condition of well being for children, adults, families 

or communities (stated in plain language). 

2. A measure that helps quantify the achievement 

of a result. 


Result 

Outcome 

Goal 

Vision 

Indicator 

Benchmark 


Population 
Total population 
Whole population 
Community-wide 

(For "client results" 
see D3 below) 


1. 

2. 


3. A coherent set of actions that has a reasoned 
chance of working (to improve results). 


Strategy 




3. 


4. A measure of how well a program, agency or service 
system service is working. 


Performance measure 
Performance indicator 


Program 

Agency 

System 


4. 


B. Other Important Ideas - Part 1 








1 . A picture of a desired future, one that is hard but 
possible to attain. 


vision 

desired future 


Often contains one 
or more results. 


1. 


2. The purpose of an organization. 


mission 

purpose 




2. 


3. A person or organization who benefits from program 
or agency service delivery. 


customer 

client 




3. 


4. A person or organization who has a significant 
interest in the performance of a program, agency 
or service system. 


stakeholder 

constituent 




4. 


5. A person or organization who has a role to play in 
improving results 


partner 


current 

potential 


5. 


6. A visual display of the history (where we've been) and 
forecast(s) (where we're headed) for a measure. 


baseline 

trendline 




6. 


7. An analysis of the conditions, causes and forces at work 
which help explain why a baseline looks the way it does. 


story behind the baseline 
epidemiology 




7. 


8. Possible actions that could make a difference 
on a result or performance measure. 


what works 

options 

strategy 


research based 
asset based 


8. 


9. A description of proposed actions. 


action plan 
stratetgic plan 
strategy 




9. 


to. The components of an action or strategic plan. 


planned accomplishments 
goals and objectives 




10 . 


1 1 . A description of the funding of existing and/or 
proposed actions. 


budget 
funding plan 




11. 


12. A document that describes what new data 
is needed or where existing data needs to be 
improved. 


data development agenda 




12. 


13. A document that describes what new information 
is needed about causes, conditions, and/or 
what works to improve results. 


information and research 
agenda 




13. 


14. A desired level of achievement for an indicator 
or performance measure 


target 

goal 

standard 


realistic 

arbitrary 

insane 


14. 













Framework Idea 


Common Labels for 
for each idea 


Motifiers - if you must 


Chosen Word or Phrase 

PICK ONE!! 


C. other Important Ideas - Part 2 








1 . A description of why we think an action or set of actions 
will work. 


Theory of change 
Logic model 


Used at both the population 
and performance levels. 


1. 


2. A structured, disciplined analysis of how well a program 
is working or has worked. 


Program Evaluation 




2. 


3. A system or process for holding people in a 
geographic area responsible for the well-being 
of the total population or some defined subpopulation. 


Results Accountability 
Outcome Accountability 
Results-based Accountability 
Outcome-based Accountability 


"Results Accountabilty" is 
sometimes used to describe 
all of 3 thru 7 combined. 


3. 


4. A system or process for holding managers and 
workers responsible for the performance of their 
programs, agencies and/or service systems 


Performance Accountability 


program 

agency 

service system 


4. 


5. A system or process of working from ends to means, 
using (population and/or program) results to drive 
decisions about what to do. 


Results-based decision making 
Outcome-based decision making 




5. 


6. A system or process of working from ends to means, 
using (population and/or program) results to drive 
the budget. 


Results-based budgeting 
Outcome-based budgeting 




6. 


7. A system or process of working from ends to means, 
using (population and/or program) results to drive 
grantmaking decisions. 


Results-based grant making 
Outcome-based grant making 




7. 


D. Types of (or ways to categorize) 
Performance Measures 








1 . Measures of the quantity or amount of effort, how 
hard did we try to deliver service, how much service 
was delivered. 


How mach did we do? 

input 

output 

resources 

process measure 

product measure 




1. 


2. Measures of the quality of effort, how well the service 
delivery and support functions were performed. 


How well did we do it? 
efficiency measure 
unit cost 
staffing ratios 
staff turnover 
staff morale 
access 

waiting time & waiting lists 
worker safety 
customer satisfaction 
process measure 




2. 


3. Measures of the quantity and quality of effect on 
customer's lives. 


Is anyone better off? 
customer outcome measure 
customer result measure 
impact measure 
effectiveness measure 
cost benefit ratio 
return on investment 
value added 
customer satisfaction 
output 
outcome 
product measure 


program or client 
program or client 


3. 


E. A BASKETFUL OF MODIFIERS 

to use with any of the above... 




Measurable Positive 

Urgent Negative 

Priority Short term 

Targeted Intermediate 

Incremental Long term 

Systemic Powdered 

Core Granulated 

Qualitative Homogenized 




















Appendix C 



Results-Based Decision Makin 



Getting from Talk to Action 
Population: e.g. Children prenatal to age 5 

Result: e.g. Children enter school healthy and 

ready to learn What we want for children in plain English, plain Spanish... 



Plus how we experience theresul 



Indicators: 

(Measures of the result) 



Baselines: 



Where we've been & p 

where we're headed o 



Target 



Trend 



Plus a Data Development Agenda 



Plus a Cost of Bad Results Analysis 



Story behind the baselines: 

The causes, the forces at work; the epidemioloqy of the baselines 



Plus Information & Research Agenda Part 



Partners with a role to play: 

Public and private sector agencies and individuals 



What works 



What would it take to turn the curve 
in this community, best practices, 

best hunches 

I Plus Information & Research Agenda Part 2 I 



Criteria 

Could include: 

Specificity: clear who, what, when, where, how 
Leverage: power to turn the curve 
Values: consistent with community values 
Reach: feasible, affordable 



Action Pian and Budget 



What we propose to do: multi-year action plan and budget 

How the "what works" pieces fit together in a community system of services and supports 

Performance Measures: Measures of how well programs, services, supports, agencies and service 
systems, included in the action plan, are working: How much did we do? How well did we do it? is anyone better 
off? ppgl 








